A Mutual Information-based Assessment of Reverse Engineering on Rewards of Reinforcement Learning

Rewards are critical hyperparameters in reinforcement learning (RL), since most cases different reward values will lead to greatly performance. Due their commercial value, RL rewards become the target of reverse engineering by inverse (IRL) algorithm family. Existing efforts typically utilize two metrics measure IRL performance: expected value difference and mean loss, which we call them EVD MRL respectively. Unfortunately, some cases, can give completely opposite results, due focusing on whole state-space while only considering partly sampled rewards. Such situation naturally rises one fundamental question: whether current assessment sufficient accurate for more general use. Thus, this paper, based metric called normalized mutual information clusters (C-NMI) propose a novel assessment; aim fill research gap middle-granularity state space between entire specific sampling space. We agglomerative nesting (AGNES) control dynamical C-NMI computing via 4-order tensor model with injected manipulated trajectories. With such model, uniformly capture different-dimension MRL, EVD, C-NMI, perform comprehensive analyses. Extensive experiments several mainstream IRLs experimented Object World, hence revealing that assessing accuracy our method increases 110.13% 116.59% respectively when compared MRL. Meanwhile, is robust than under demonstrations.